dataeval.extractors.UncertaintyExtractor¶

class dataeval.extractors.UncertaintyExtractor(scores, preds_type='logits', normalize=True)¶

Per-instance prediction entropy as a drift feature.

Implements the FeatureExtractor protocol: __call__ returns a (n_samples, 1) array of Shannon-entropy uncertainty scores, suitable for DriftUnivariate.

Parameters:¶

scores : FeatureExtractor¶

Producer of per-instance class scores (n, n_classes). Owns the model, backend, batching and any detection decoding.

preds_type : "probs" or "logits", default "logits"¶

Format of the scores. “logits” applies softmax before entropy; “probs” expects values that already sum to 1.

normalize : bool, default True¶

Normalize Shannon entropy by the maximum possible entropy for the number of classes present.

Example

>>> import numpy as np
>>> from dataeval.extractors._uncertainty import UncertaintyExtractor
>>>
>>> class FixedScores:
...     def __call__(self, data):
...         return np.array([[2.0, 1.0, 0.0], [1.0, 1.0, 1.0]], dtype=np.float32)
>>>
>>> ex = UncertaintyExtractor(FixedScores(), preds_type="logits")
>>> out = ex(None)
>>> out.shape
(2, 1)