dataeval.shift.DriftMMD¶

class dataeval.shift.DriftMMD(data, p_val=None, update_strategy=None, sigma=None, n_permutations=None, permutation_batch_size=None, device=None, config=None)¶

Drift detector using Maximum Mean Discrepancy (MMD) Drift Detection with permutation test.

Detects distributional differences by comparing kernel embeddings of reference and test datasets in a reproducing kernel Hilbert space (RKHS). Uses permutation testing to assess statistical significance of the observed MMD^2 statistic.

MMD is particularly effective for high-dimensional data like images as it can capture complex distributional differences that univariate tests might miss. The kernel-based approach enables detection of both marginal and dependency changes between features.

Parameters:¶

data : Array¶

Reference dataset used as baseline distribution for drift detection. Should represent the expected data distribution.

p_val : float, default 0.05¶

Significance threshold for statistical tests, between 0 and 1. For FDR correction, this represents the acceptable false discovery rate. Default 0.05 provides 95% confidence level for drift detection.

update_strategy : UpdateStrategy or None, default None¶

Strategy for updating reference data when new data arrives. When None, reference data remains fixed throughout detection.

sigma : Array or None, default None¶

Bandwidth parameter(s) for the Gaussian RBF kernel. Controls the kernel’s sensitivity to distance between data points. When None, automatically selects bandwidth using median heuristic. Can provide multiple values as array to average over different scales.

n_permutations : int, default 100¶

Number of random permutations used in the permutation test to estimate the null distribution of MMD² under no drift. Higher values provide more accurate p-value estimates but increase computation time. Default 100 balances statistical accuracy with computational efficiency.

permutation_batch_size : int or "auto", default "auto"¶

Batch size for computing permutations to reduce memory usage. When “auto” (default), automatically detects appropriate batch size based on available GPU memory (on CUDA devices) or computes all permutations at once (on CPU). Set to an integer to manually specify batch size. Useful when working with large kernel matrices or many permutations to avoid GPU out-of-memory errors. For example, with n_permutations=100 and permutation_batch_size=10, permutations are computed in 10 batches of 10 each.

device : DeviceLike or None, default None¶

Hardware device for computation. When None, automatically selects DataEval’s configured device, falling back to PyTorch’s default.

config : DriftMMD.Config or None, default None¶

Optional configuration object with default parameters. Parameters specified directly in __init__ will override config defaults.

p_val¶

Significance threshold for statistical tests.

Type:¶

float

update_strategy¶

Reference data update strategy.

Type:¶

UpdateStrategy or None

n¶

Number of samples in the reference dataset.

Type:¶

int

sigma¶

Gaussian RBF kernel bandwidth parameter(s).

Type:¶

Array or None

n_permutations¶

Number of permutations for statistical testing.

Type:¶

int

permutation_batch_size¶

Batch size for computing permutations, or “auto” for automatic detection.

Type:¶

int or “auto”

device¶

Hardware device used for computations.

Type:¶

torch.device

Example

Initialize with image embeddings

>>> train_emb = np.ones((100, 128), dtype=np.float32)
>>> drift = DriftMMD(train_emb)

Test incoming images for drift

>>> test_emb = np.zeros((20, 128), dtype=np.float32)
>>> result = drift.predict(test_emb)

>>> print(f"Drift detected: {result.drifted}")
Drift detected: True

>>> print(f"Mean MMD statistic: {result.distance:.2f}")
Mean MMD statistic: 1.26

Using configuration:

>>> config = DriftMMD.Config(p_val=0.01, n_permutations=200)
>>> drift = DriftMMD(train_emb, config=config)

predict(data)¶

Predict whether a batch of data has drifted from the reference data.

Then updates reference data using specified strategy.

Parameters:¶

data : Array¶

Batch of instances to predict drift on.

Returns:¶

Output class containing the drift prediction, p-value, threshold and MMD metric.

Return type:¶

DriftMMDOutput

score(data)¶

Compute the p-value resulting from a permutation test using the maximum mean discrepancy.

The maximum mean discrepancy is used as a distance measure between the reference data and the data to be tested.

Parameters:¶

data : Array¶

Batch of instances to score.

Returns:¶

p-value obtained from the permutation test, MMD^2 between the reference and test set, and MMD^2 threshold above which drift is flagged

Return type:¶

tuple(float, float, float)

property x_ref : numpy.typing.NDArray[numpy.float32]¶

Reference data for drift detection.

Lazily encodes the reference dataset on first access. Data is flattened and converted to 32-bit floating point for consistent numerical processing across different input types.

Returns:¶

Reference data as flattened 32-bit floating point array. Shape is (n_samples, n_features_flattened).

Return type:¶

NDArray[np.float32]

Notes

Data is cached after first access to avoid repeated encoding overhead.

Classes¶