Dataset Linting Tutorial#

Problem Statement#

Exploratory data analysis (EDA) can be overwhelming. There are so many things to check. Duplicates in your dataset, bad/corrupted images in the set, blurred or bright/dark images, the list goes on.

DataEval created a Linting class to assist you with your EDA so you can start training your models on high quality data.

When to use#

The Linting class should be used during the initial EDA process or if you are trying to verify that you have the right data in your dataset.

What you will need#

A dataset to analyze
A python environment with the following packages installed:
- dataeval[torch] or dataeval[all]
- torchvision

Getting Started#

Let’s import the required libraries needed to set up a minimal working example

import numpy as np
import torch
import torchvision.datasets as datasets
import torchvision.transforms.v2 as v2

from dataeval.detectors.linters import Outliers

Loading in the data#

We are going to start by loading in torchvision’s CIFAR-10 dataset.

The CIFAR-10 dataset contains 60,000 images - 50,000 in the train set and 10,000 in the test set. For the purposes of this demonstration, we are just going to use the test set.

# Load in the cifar-10 dataset from torchvision
to_tensor = v2.Compose([v2.ToImage(), v2.ToDtype(torch.float32, scale=True)])
testing_dataset = datasets.CIFAR10("./data", train=False, download=True, transform=to_tensor)
test_data = np.array(testing_dataset.data, dtype=float)

Files already downloaded and verified

Linting the Dataset#

Now we can begin finding those images which are significantly different from the rest of the data.

# Initialize the Duplicates class
outliers = Outliers()

# Evaluate the data
results = outliers.evaluate(test_data)

The results are a dictionary with the keys being the image that has an issue in one of the listed properties below:

Brightness
Blurriness
Missing
Zero
Width
Height
Size
Aspect Ratio
Channels
Depth

print(f"Total number of images with an issue: {len(results.issues)}")

Total number of images with an issue: 673

# Show each image that has at least one issue
for image, issue in results.issues.items():
    print(f"{image} - {issue}")

- {'zero': 0.61}
- {'zero': 1.33}
- {'zero': 6.69}
- {'blurriness': 95.75}
- {'zero': 0.53}
- {'blurriness': 112.38}
- {'zero': 3.29}
- {'blurriness': 93.06}
- {'zero': 1.45}
- {'blurriness': 91.88}
- {'zero': 0.84}
- {'blurriness': 103.75}
- {'zero': 5.89}
- {'blurriness': 114.44}
- {'zero': 1.0}
- {'blurriness': 91.44}
- {'blurriness': 107.44}
- {'blurriness': 90.56}
- {'zero': 3.42}
- {'brightness': 0.93}
- {'blurriness': 111.0}
- {'zero': 1.29}
- {'blurriness': 93.12}
- {'zero': 1.23}
- {'zero': 0.69}
- {'zero': 0.7}
- {'zero': 0.44}
- {'blurriness': 103.31}
- {'blurriness': 110.56}
- {'brightness': 0.91}
- {'blurriness': 115.0}
- {'blurriness': 107.75}
- {'blurriness': 100.12}
- {'zero': 0.44}
- {'zero': 1.86}
- {'blurriness': 92.5}
- {'blurriness': 104.12}
- {'blurriness': 91.62}
- {'zero': 0.82}
- {'blurriness': 96.62}
- {'zero': 0.58}
- {'zero': 5.31}
- {'zero': 0.52}
- {'blurriness': 94.81}
- {'blurriness': 92.0}
- {'zero': 1.17}
- {'zero': 0.53}
- {'blurriness': 113.0}
- {'blurriness': 90.56}
- {'blurriness': 106.25}
- {'blurriness': 110.56}
- {'blurriness': 107.25}
- {'blurriness': 107.75}
- {'zero': 4.89}
- {'zero': 3.01}
- {'blurriness': 114.12}
- {'zero': 0.48}
- {'zero': 11.01}
- {'blurriness': 109.25}
- {'zero': 0.57}
- {'zero': 0.99}
- {'zero': 1.23}
- {'blurriness': 100.88}
- {'blurriness': 99.44}
- {'zero': 0.5}
- {'blurriness': 103.19}
- {'zero': 0.98}
- {'brightness': 0.92}
- {'zero': 4.05}
- {'blurriness': 100.69}
- {'blurriness': 92.0}
- {'zero': 1.57}
- {'zero': 0.66}
- {'zero': 3.92}
- {'brightness': 0.88}
- {'blurriness': 95.62}
- {'zero': 0.57}
- {'blurriness': 96.06}
- {'blurriness': 119.81}
- {'blurriness': 103.69, 'zero': 0.89}
- {'zero': 0.82}
- {'zero': 0.56}
- {'blurriness': 104.5}
- {'blurriness': 92.38}
- {'blurriness': 98.38}
- {'blurriness': 111.69, 'zero': 0.94}
- {'zero': 0.48}
- {'blurriness': 101.62}
- {'blurriness': 102.19}
- {'blurriness': 106.56}
- {'blurriness': 111.88}
- {'zero': 0.78}
- {'blurriness': 103.31}
- {'blurriness': 98.19}
- {'zero': 0.74}
- {'zero': 2.96}
- {'blurriness': 98.44}
- {'blurriness': 111.88}
- {'brightness': 0.92}
- {'zero': 0.49}
- {'zero': 0.5}
- {'brightness': 0.88}
- {'zero': 0.51}
- {'zero': 1.69}
- {'zero': 0.53}
- {'zero': 0.57}
- {'blurriness': 105.12}
- {'blurriness': 93.19}
- {'zero': 0.82}
- {'zero': 0.43}
- {'blurriness': 92.38}
- {'zero': 1.0}
- {'zero': 1.51}
- {'zero': 0.5}
- {'blurriness': 93.56}
- {'blurriness': 104.38, 'zero': 0.57}
- {'zero': 2.9}
- {'blurriness': 96.88}
- {'blurriness': 106.62}
- {'zero': 1.83}
- {'blurriness': 105.19, 'zero': 1.1}
- {'blurriness': 106.12}
- {'zero': 0.95}
- {'zero': 9.6}
- {'blurriness': 105.94}
- {'blurriness': 102.06}
- {'zero': 0.82}
- {'blurriness': 101.44}
- {'zero': 0.78}
- {'blurriness': 115.06}
- {'blurriness': 119.38}
- {'blurriness': 99.69}
- {'zero': 0.61}
- {'blurriness': 120.19}
- {'blurriness': 114.31}
- {'blurriness': 109.25}
- {'blurriness': 94.38}
- {'zero': 1.89}
- {'blurriness': 110.81}
- {'blurriness': 117.62}
- {'blurriness': 96.81}
- {'blurriness': 95.69}
- {'blurriness': 104.94, 'zero': 0.77}
- {'blurriness': 96.25}
- {'zero': 4.89}
- {'zero': 1.26}
- {'blurriness': 120.19}
- {'brightness': 0.9}
- {'zero': 0.44}
- {'blurriness': 119.62}
- {'blurriness': 96.06}
- {'blurriness': 97.62}
- {'zero': 0.81}
- {'blurriness': 98.12}
- {'blurriness': 90.69, 'zero': 0.44}
- {'zero': 0.46}
- {'blurriness': 92.75}
- {'blurriness': 97.25}
- {'blurriness': 116.62}
- {'zero': 3.01}
- {'blurriness': 103.12}
- {'zero': 0.89}
- {'blurriness': 105.69}
- {'blurriness': 103.44}
- {'zero': 0.44}
- {'zero': 0.46}
- {'zero': 1.5}
- {'zero': 8.05}
- {'zero': 3.75}
- {'blurriness': 92.88}
- {'zero': 3.76}
- {'zero': 2.93}
- {'blurriness': 110.44}
- {'zero': 0.74}
- {'blurriness': 115.44}
- {'blurriness': 115.12}
- {'blurriness': 120.19, 'zero': 0.95}
- {'blurriness': 111.38}
- {'zero': 0.85}
- {'blurriness': 105.31}
- {'blurriness': 106.56}
- {'blurriness': 101.19}
- {'zero': 15.43}
- {'blurriness': 93.31}
- {'zero': 1.41}
- {'zero': 0.8}
- {'blurriness': 102.81}
- {'zero': 3.79}
- {'zero': 0.58}
- {'blurriness': 104.0}
- {'zero': 1.91}
- {'blurriness': 118.56}
- {'zero': 0.98}
- {'blurriness': 104.81}
- {'blurriness': 100.5}
- {'blurriness': 92.0}
- {'blurriness': 114.5}
- {'zero': 0.92}
- {'blurriness': 110.19}
- {'zero': 1.27}
- {'zero': 2.38}
- {'blurriness': 92.12}
- {'zero': 0.76}
- {'zero': 0.53}
- {'zero': 2.1}
- {'blurriness': 94.31}
- {'blurriness': 105.19}
- {'zero': 1.0}
- {'blurriness': 105.31}
- {'blurriness': 112.81}
- {'blurriness': 119.81}
- {'blurriness': 110.44}
- {'blurriness': 93.5}
- {'blurriness': 93.19}
- {'blurriness': 99.19}
- {'blurriness': 96.19}
- {'zero': 0.81}
- {'zero': 1.19}
- {'zero': 0.5}
- {'blurriness': 117.12}
- {'blurriness': 101.56}
- {'blurriness': 105.25}
- {'blurriness': 114.62}
- {'zero': 0.45}
- {'blurriness': 98.88}
- {'brightness': 0.99}
- {'zero': 0.47}
- {'blurriness': 109.19}
- {'zero': 1.33}
- {'zero': 1.24}
- {'blurriness': 104.06}
- {'blurriness': 100.94}
- {'blurriness': 95.62}
- {'brightness': 0.92}
- {'zero': 0.78}
- {'blurriness': 105.44}
- {'zero': 15.32}
- {'blurriness': 98.94}
- {'blurriness': 103.81}
- {'zero': 0.6}
- {'zero': 0.68}
- {'blurriness': 120.0}
- {'blurriness': 109.88}
- {'zero': 1.81}
- {'blurriness': 90.62}
- {'blurriness': 101.12}
- {'zero': 1.99}
- {'blurriness': 99.38}
- {'zero': 0.53}
- {'blurriness': 93.0}
- {'blurriness': 100.44}
- {'zero': 0.46}
- {'blurriness': 103.44}
- {'blurriness': 99.62}
- {'brightness': 0.91}
- {'blurriness': 109.44}
- {'blurriness': 105.81}
- {'blurriness': 91.0}
- {'blurriness': 91.25}
- {'zero': 0.47}
- {'zero': 0.45}
- {'blurriness': 97.5}
- {'zero': 3.45}
- {'blurriness': 110.88}
- {'zero': 1.15}
- {'zero': 1.85}
- {'zero': 0.53}
- {'zero': 0.97}
- {'zero': 1.62}
- {'blurriness': 112.44}
- {'zero': 0.82}
- {'zero': 1.12}
- {'blurriness': 114.31}
- {'zero': 0.51}
- {'blurriness': 93.56}
- {'blurriness': 101.5}
- {'blurriness': 98.44}
- {'blurriness': 91.25}
- {'blurriness': 110.81}
- {'blurriness': 120.19}
- {'blurriness': 96.75}
- {'blurriness': 91.12}
- {'blurriness': 109.25}
- {'blurriness': 100.12}
- {'blurriness': 108.38}
- {'blurriness': 90.94}
- {'zero': 0.62}
- {'blurriness': 119.88}
- {'zero': 9.65}
- {'blurriness': 94.5}
- {'zero': 0.74}
- {'blurriness': 96.81}
- {'zero': 0.66}
- {'zero': 1.51}
- {'zero': 0.68}
- {'zero': 4.43}
- {'zero': 2.32}
- {'zero': 0.6}
- {'blurriness': 105.06}
- {'brightness': 0.88}
- {'blurriness': 107.75}
- {'blurriness': 107.69}
- {'blurriness': 96.94}
- {'zero': 0.72}
- {'blurriness': 105.44, 'zero': 0.73}
- {'blurriness': 93.69, 'zero': 1.09}
- {'blurriness': 117.44}
- {'blurriness': 119.12}
- {'zero': 1.11}
- {'blurriness': 111.88}
- {'brightness': 0.92}
- {'zero': 0.64}
- {'blurriness': 101.5}
- {'brightness': 0.88}
- {'blurriness': 103.75}
- {'blurriness': 120.19}
- {'zero': 9.69}
- {'blurriness': 99.25}
- {'zero': 2.23}
- {'blurriness': 99.06}
- {'blurriness': 104.06}
- {'blurriness': 100.5}
- {'zero': 0.92}
- {'zero': 0.44}
- {'zero': 3.83}
- {'blurriness': 101.25}
- {'blurriness': 113.62}
- {'zero': 1.17}
- {'zero': 0.77}
- {'blurriness': 92.94}
- {'blurriness': 92.0}
- {'blurriness': 120.19}
- {'zero': 0.48}
- {'blurriness': 119.12}
- {'blurriness': 100.94}
- {'zero': 0.54}
- {'brightness': 0.89}
- {'blurriness': 108.19}
- {'blurriness': 110.69}
- {'blurriness': 119.31}
- {'zero': 1.83}
- {'blurriness': 116.06}
- {'blurriness': 100.56}
- {'zero': 0.7}
- {'zero': 4.76}
- {'zero': 0.7}
- {'zero': 0.71}
- {'blurriness': 100.81}
- {'zero': 1.15}
- {'zero': 0.96}
- {'zero': 1.76}
- {'blurriness': 98.06}
- {'blurriness': 95.0}
- {'zero': 0.59}
- {'blurriness': 117.75}
- {'blurriness': 93.0}
- {'zero': 1.72}
- {'zero': 0.73}
- {'blurriness': 92.44}
- {'zero': 4.31}
- {'blurriness': 93.88}
- {'zero': 7.5}
- {'blurriness': 113.31}
- {'blurriness': 119.0}
- {'blurriness': 92.75}
- {'zero': 0.51}
- {'blurriness': 104.31}
- {'blurriness': 107.88}
- {'blurriness': 111.0}
- {'blurriness': 117.56}
- {'blurriness': 103.19}
- {'zero': 3.81}
- {'zero': 10.62}
- {'blurriness': 91.56}
- {'zero': 0.66}
- {'zero': 2.76}
- {'zero': 1.02}
- {'blurriness': 98.75}
- {'zero': 0.95}
- {'zero': 0.49}
- {'zero': 0.53}
- {'zero': 10.96}
- {'brightness': 0.9}
- {'zero': 3.23}
- {'blurriness': 116.31}
- {'blurriness': 92.06}
- {'blurriness': 120.19}
- {'blurriness': 99.69}
- {'zero': 0.67}
- {'blurriness': 91.31}
- {'blurriness': 92.5}
- {'blurriness': 95.31}
- {'blurriness': 112.25, 'zero': 2.83}
- {'blurriness': 94.69}
- {'blurriness': 120.19}
- {'zero': 3.67}
- {'blurriness': 104.81}
- {'zero': 0.47}
- {'blurriness': 105.88}
- {'blurriness': 96.31}
- {'blurriness': 91.56}
- {'zero': 1.76}
- {'zero': 0.79}
- {'blurriness': 112.0}
- {'zero': 2.67}
- {'zero': 4.22}
- {'blurriness': 102.69}
- {'blurriness': 91.38}
- {'blurriness': 109.62}
- {'blurriness': 98.19}
- {'zero': 1.43}
- {'blurriness': 95.25}
- {'zero': 1.92}
- {'zero': 1.44}
- {'brightness': 0.9}
- {'zero': 0.58}
- {'zero': 3.48}
- {'blurriness': 109.0}
- {'zero': 1.02}
- {'blurriness': 102.81}
- {'blurriness': 91.62}
- {'blurriness': 103.25}
- {'blurriness': 101.31}
- {'zero': 11.85}
- {'blurriness': 91.75}
- {'blurriness': 107.5}
- {'blurriness': 113.75}
- {'zero': 0.49}
- {'zero': 0.78}
- {'zero': 1.58}
- {'blurriness': 111.88}
- {'blurriness': 97.12}
- {'zero': 0.93}
- {'blurriness': 102.56}
- {'zero': 3.25}
- {'zero': 3.63}
- {'blurriness': 101.62}
- {'brightness': 0.9}
- {'zero': 7.61}
- {'blurriness': 105.5}
- {'blurriness': 97.31}
- {'blurriness': 114.62, 'zero': 1.34}
- {'blurriness': 90.81}
- {'zero': 2.94}
- {'blurriness': 119.56}
- {'zero': 0.89}
- {'blurriness': 97.75}
- {'zero': 8.04}
- {'blurriness': 111.12}
- {'blurriness': 100.12}
- {'zero': 0.7}
- {'blurriness': 102.44}
- {'zero': 0.76}
- {'blurriness': 102.88}
- {'blurriness': 95.25}
- {'blurriness': 113.88}
- {'zero': 0.96}
- {'blurriness': 114.88}
- {'blurriness': 112.31}
- {'zero': 8.09}
- {'blurriness': 95.19}
- {'blurriness': 101.94}
- {'zero': 1.11}
- {'zero': 0.67}
- {'blurriness': 95.75}
- {'blurriness': 110.38}
- {'zero': 1.39}
- {'zero': 0.6}
- {'zero': 5.12}
- {'zero': 2.5}
- {'blurriness': 101.12}
- {'blurriness': 92.44}
- {'blurriness': 110.25}
- {'blurriness': 102.31}
- {'zero': 2.53}
- {'blurriness': 110.88}
- {'blurriness': 109.69}
- {'blurriness': 109.0}
- {'blurriness': 98.5}
- {'blurriness': 90.69}
- {'blurriness': 99.75}
- {'blurriness': 93.31}
- {'blurriness': 100.0}
- {'zero': 0.61}
- {'blurriness': 91.94}
- {'blurriness': 104.31}
- {'zero': 5.21}
- {'blurriness': 95.06}
- {'zero': 1.46}
- {'blurriness': 94.12}
- {'blurriness': 91.5}
- {'blurriness': 103.06}
- {'blurriness': 95.31, 'zero': 0.55}
- {'brightness': 0.89}
- {'blurriness': 96.12}
- {'zero': 8.78}
- {'blurriness': 98.44}
- {'blurriness': 114.81}
- {'blurriness': 101.88}
- {'blurriness': 106.44}
- {'blurriness': 112.94}
- {'blurriness': 108.94}
- {'zero': 0.58}
- {'blurriness': 99.19}
- {'blurriness': 108.38}
- {'zero': 2.34}
- {'blurriness': 115.62}
- {'zero': 1.7}
- {'blurriness': 99.56}
- {'zero': 1.24}
- {'zero': 0.61}
- {'blurriness': 104.75}
- {'blurriness': 116.62}
- {'blurriness': 105.56}
- {'blurriness': 107.94, 'zero': 2.1}
- {'zero': 0.56}
- {'blurriness': 113.25}
- {'blurriness': 92.44}
- {'zero': 0.93}
- {'blurriness': 104.31}
- {'blurriness': 96.12}
- {'blurriness': 102.25}
- {'zero': 0.45}
- {'blurriness': 99.38}
- {'zero': 0.46}
- {'blurriness': 120.19}
- {'blurriness': 95.5}
- {'zero': 1.67}
- {'zero': 13.48}
- {'brightness': 0.93}
- {'blurriness': 96.88}
- {'brightness': 0.89}
- {'blurriness': 102.19}
- {'blurriness': 101.56}
- {'zero': 0.84}
- {'zero': 0.78}
- {'zero': 0.85}
- {'zero': 0.99}
- {'blurriness': 105.19}
- {'zero': 1.31}
- {'blurriness': 96.44}
- {'blurriness': 96.56}
- {'zero': 1.77}
- {'blurriness': 94.38}
- {'blurriness': 99.38}
- {'blurriness': 92.5}
- {'zero': 0.53}
- {'brightness': 0.88}
- {'zero': 0.61}
- {'blurriness': 96.06}
- {'zero': 1.53}
- {'blurriness': 102.62}
- {'blurriness': 118.62}
- {'blurriness': 108.81}
- {'zero': 5.01}
- {'blurriness': 91.31}
- {'blurriness': 118.88}
- {'blurriness': 93.81}
- {'blurriness': 99.0}
- {'zero': 0.43}
- {'blurriness': 114.56}
- {'zero': 1.08}
- {'blurriness': 90.56}
- {'zero': 0.51}
- {'zero': 7.43}
- {'blurriness': 110.44}
- {'zero': 2.38}
- {'zero': 3.87}
- {'blurriness': 117.69}
- {'zero': 1.21}
- {'zero': 0.46}
- {'blurriness': 97.75}
- {'blurriness': 114.75}
- {'zero': 0.48}
- {'zero': 0.75}
- {'blurriness': 106.44}
- {'zero': 0.47}
- {'zero': 5.96}
- {'zero': 1.59}
- {'blurriness': 118.44}
- {'zero': 0.48}
- {'zero': 9.45}
- {'zero': 9.85}
- {'zero': 0.73}
- {'blurriness': 96.69}
- {'zero': 1.38}
- {'zero': 2.99}
- {'blurriness': 95.94}
- {'blurriness': 105.0}
- {'blurriness': 90.94}
- {'zero': 0.7}
- {'blurriness': 104.19}
- {'zero': 4.06}
- {'zero': 0.46}
- {'zero': 0.69}
- {'blurriness': 103.38}
- {'zero': 0.45}
- {'zero': 4.54}
- {'zero': 0.43}
- {'blurriness': 111.0}
- {'brightness': 0.94}
- {'blurriness': 111.56}
- {'blurriness': 111.69}
- {'zero': 0.71}
- {'blurriness': 91.19}
- {'blurriness': 112.94}
- {'blurriness': 114.75}
- {'blurriness': 99.31}
- {'blurriness': 98.44}
- {'blurriness': 118.81}
- {'blurriness': 99.38}
- {'blurriness': 120.19}
- {'blurriness': 99.19}
- {'zero': 0.47}
- {'zero': 1.21}
- {'blurriness': 94.94}
- {'blurriness': 98.69}
- {'zero': 0.69}
- {'blurriness': 103.31}
- {'blurriness': 107.0}
- {'zero': 16.05}
- {'zero': 1.12}
- {'zero': 1.46}
- {'blurriness': 115.44}
- {'zero': 2.57}
- {'zero': 6.41}
- {'zero': 0.61}
- {'zero': 0.53}
- {'zero': 5.73}
- {'zero': 7.57}
- {'blurriness': 108.38}
- {'blurriness': 98.5}
- {'zero': 0.55}
- {'blurriness': 116.44}
- {'blurriness': 92.38}
- {'blurriness': 107.62, 'zero': 4.2}
- {'zero': 0.5}
- {'zero': 0.54}
- {'zero': 7.39}
- {'blurriness': 92.19}
- {'blurriness': 108.81}
- {'zero': 0.62}
- {'blurriness': 108.81}
- {'blurriness': 114.25}
- {'blurriness': 96.94}
- {'zero': 0.82}
- {'blurriness': 92.81}
- {'zero': 0.56}
- {'zero': 2.79}
- {'zero': 0.81}
- {'zero': 0.91}
- {'blurriness': 101.12}
- {'zero': 8.78}
- {'zero': 5.76}
- {'blurriness': 111.81}
- {'blurriness': 96.88}
- {'blurriness': 102.25}
- {'blurriness': 101.38, 'zero': 0.57}
- {'blurriness': 91.0}
- {'zero': 3.49}
- {'blurriness': 107.94}
- {'brightness': 0.93}
- {'blurriness': 109.81}
- {'blurriness': 94.69}
- {'zero': 4.62}
- {'zero': 0.94}
- {'brightness': 0.94}
- {'zero': 0.67}
- {'blurriness': 100.44}
- {'blurriness': 97.38}
- {'zero': 0.86}
- {'zero': 0.89}
- {'blurriness': 106.25}