dataeval.core.uap¶

dataeval.core.uap(labels, scores)¶

Estimate the empirical mean precision for the upperbound average precision.

Warning

This feature is experimental and may change or be removed in future releases.

Uses the FR Test Statistic based approach.

Parameters:¶

labels : ArrayLike¶

A 2D array of n_samples of class labels with M unique classes.

scores : ArrayLike¶

A 2D array of class probabilities per image.

Returns:¶

The empirical mean precision estimate.

Return type:¶

float

Raises:¶

ValueError – If unique classes M < 2.

Notes

This function calculates the empirical mean precision using the average_precision_score from scikit-learn, weighted by the class distribution.

Examples

>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> uap(y_true, y_scores)
0.8333333333333333

>>> y_true = np.array([0, 0, 1, 1, 2, 2])
>>> y_scores = np.array(
...     [
...         [0.7, 0.2, 0.1],
...         [0.4, 0.3, 0.3],
...         [0.1, 0.8, 0.1],
...         [0.2, 0.3, 0.5],
...         [0.4, 0.4, 0.2],
...         [0.1, 0.2, 0.7],
...     ]
... )
>>> uap(y_true, y_scores)
0.7777777777777777