dataeval.core.uap¶

dataeval.core.uap(labels, scores)¶

Estimate the empirical mean precision for the upperbound average precision.

Uses the FR Test Statistic based approach.

Parameters:¶

labels : ArrayLike¶

A 2D array of n_samples of class labels with M unique classes.

scores : ArrayLike¶

A 2D array of class probabilities per image.

Returns:¶

The empirical mean precision estimate.

Return type:¶

float

Raises:¶

ValueError – If unique classes M < 2.

Notes

This function calculates the empirical mean precision using the average_precision_score from scikit-learn, weighted by the class distribution.

Examples

>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> uap(y_true, y_scores)
0.8333333333333333

>>> y_true = np.array([0, 0, 1, 1, 2, 2])
>>> y_scores = np.array(
...     [
...         [0.7, 0.2, 0.1],
...         [0.4, 0.3, 0.3],
...         [0.1, 0.8, 0.1],
...         [0.2, 0.3, 0.5],
...         [0.4, 0.4, 0.2],
...         [0.1, 0.2, 0.7],
...     ]
... )
>>> uap(y_true, y_scores)
0.7777777777777777