Parity¶
What is it¶
Parity is a means for assessing fairness in machine learning by looking for statistical independence between metadata factors and class labels in a dataset. This assessment helps a user understand sources of potential bias before a model gets trained on it and inadvertently learns spurious correlations. In an ideal case with zero bias, the probability of observing a class label would be independent from observing a specific metadata factor.
When to use it¶
For both model and dataset development it is important to understand correlational relationships that underlie the dataset. Often, opportunities for data collection are sparse, available only in non-operational locations and conditions, with limited target diversity, etc. A model trained on these realistic datasets could learn to use secondary information to perform the primary learning task, reducing the model’s ability to generalize to new domains or to perform unexpectedly when presented with new data. Parity metric provides a method for identifying linear relationships between dataset factors and class labels a priori. A T&E engineer or model developer should then use that information to design tests for model generalization or data augmentation to mitigate the opportunity for shortcut learning or sampling imbalance.
In order to use parity, the user must supply their metadata in
a DataEval specific format. Because of this requirement, DataEval has a
Metadata class that will take in user metadata and format it
into DataEval’s format. The parity function takes in the Metadata class for
its analysis.
Why use parity over other statistical methods?¶
Parity measures bias on the dataset prior to model testing allowing for faster iterations in developing unbiased ML pipelines. Several methods such as error rate balances, test-fairness, positive/negative class balance, and equal-confusion fairness, which are commonly used for assessing bias and fairness, are calculated based on model predictions and probabilities. Thus, those methods have to be evaluated after a model is already trained.
What can be done with the parity information?¶
If all metadata factors are independent from labels, a model trained on it will be less likely to overfit to spurious correlations.
If a metadata factor is not independent from class labels, then a model trained on the dataset could exhibit unintended bias. In this case, action is recommended. Actions include, but are not limited to:
Collecting or generating additional training data that has consistent label distributions across all values of the metadata factor.
Identifying how the spurious correlation manifests in the embeddings in a model, and subtracting out the bias in latent space.
Assigning weights to the loss function that de-emphasize samples that exhibit spurious correlations.