dataeval.protocols¶

Common type protocols used for interoperability with DataEval.

Attributes¶

`ArrayLike`	Type alias for a Union representing objects that can be coerced into an array.
`DeviceLike`	Type alias for a Union representing types that specify a torch.device.
`ImageClassificationDataset`	Type alias for an `AnnotatedDataset` of `ImageClassificationDatum` elements.
`ImageClassificationDatum`	Type alias for an image classification datum tuple.
`ObjectDetectionDataset`	Type alias for an `AnnotatedDataset` of `ObjectDetectionDatum` elements.
`ObjectDetectionDatum`	Type alias for an object detection datum tuple.
`SegmentationDataset`	Type alias for an `AnnotatedDataset` of `SegmentationDatum` elements.
`SegmentationDatum`	Type alias for a segmentation datum tuple.
`ThresholdLike`	Type alias for threshold specifications.

Classes¶

`AnnotatedDataset`	Protocol for a generic AnnotatedDataset.
`Array`	Protocol for array objects providing interoperability with DataEval.
`Chunker`	Protocol for chunking datasets into subsets by returning index arrays.
`Dataset`	Protocol for a generic Dataset.
`EvaluationSchedule`	Protocol for determining evaluation points in sufficiency analysis.
`EvaluationStrategy`	Protocol defining the interface for evaluating a trained model.
`EvidenceLowerBoundLossFn`	Protocol for Evidence Lower Bound (ELBO) loss functions.
`FeatureExtractor`	Protocol defining a feature extraction function for drift detection.
`LossFn`	Protocol for generic loss functions that can be used with PyTorch models.
`MetadataLike`	Minimal protocol for metadata objects used in bias and quality analysis.
`ModelResetStrategy`	Protocol for resetting model parameters between training runs.
`ProgressCallback`	Protocol for a callable progress callback function.
`ReconstructionLossFn`	Protocol for reconstruction-based loss functions (Autoencoder).
`SegmentationTarget`	Protocol for targets in a Segmentation dataset.
`SequenceLike`	Protocol for sequence-like objects that can be indexed and iterated.
`Threshold`	Protocol for threshold objects used in bias and quality evaluators.
`TrainingStrategy`	Protocol defining the interface for training a model on a dataset subset.
`Transform`	Protocol defining a transform function.
`UpdateStrategy`	Protocol defining the interface for updating reference data in drift detectors.

Module Contents¶

type dataeval.protocols.ArrayLike = np.typing.ArrayLike¶: Type alias for a Union representing objects that can be coerced into an array.

See also

NumPy ArrayLike

type dataeval.protocols.DeviceLike = int | str | tuple[str, int] | torch.device¶: Type alias for a Union representing types that specify a torch.device.

See also

torch.device

type dataeval.protocols.ImageClassificationDataset = AnnotatedDataset[ImageClassificationDatum]¶: Type alias for an AnnotatedDataset of ImageClassificationDatum elements.

type dataeval.protocols.ImageClassificationDatum = tuple[ArrayLike, ArrayLike, DatumMetadata]¶

Type alias for an image classification datum tuple.

ArrayLike of shape (C, H, W) - Image data in channel, height, width format.
ArrayLike of shape (N,) - Class label as one-hot encoded ground-truth or prediction confidences.
dict[str, Any] - Datum level metadata.

type dataeval.protocols.ObjectDetectionDataset = AnnotatedDataset[ObjectDetectionDatum]¶: Type alias for an AnnotatedDataset of ObjectDetectionDatum elements.

type dataeval.protocols.ObjectDetectionDatum = tuple[ArrayLike, ObjectDetectionTarget, DatumMetadata]¶

Type alias for an object detection datum tuple.

ArrayLike of shape (C, H, W) - Image data in channel, height, width format.
ObjectDetectionTarget - Object detection target information for the image.
dict[str, Any] - Datum level metadata.

type dataeval.protocols.SegmentationDataset = AnnotatedDataset[SegmentationDatum]¶: Type alias for an AnnotatedDataset of SegmentationDatum elements.

type dataeval.protocols.SegmentationDatum = tuple[ArrayLike, SegmentationTarget, DatumMetadata]¶

Type alias for a segmentation datum tuple.

ArrayLike of shape (C, H, W) - Image data in channel, height, width format.
SegmentationTarget - Segmentation target information for the image.
dict[str, Any] - Datum level metadata.

dataeval.protocols.ThresholdLike¶

Type alias for threshold specifications.

Values default to modified z-score thresholds if not provided. - float: symmetric multiplier (same for lower and upper) - str: named threshold (e.g., “modzscore”) with default bounds - tuple[float | None, float | None]: (lower, upper) for asymmetric bounds - tuple[str, float | tuple[float | None, float | None]]: named threshold with optional lower and upper bounds - tuple[str, bounds, (lower_limit, upper_limit)]: named threshold with bounds and limit clamping, e.g. ("zscore", (1.0, 3.5), (0.0, 1.0)). Pass None for bounds to use defaults: ("zscore", None, (0.0, 1.0)) - tuple[bounds | None, (lower_limit, upper_limit)]: default threshold with bounds and limit clamping, e.g. (2.5, (0.0, 1.0)) or (None, (0.0, 1.0)) for default multiplier - Threshold: a fully configured Threshold instance