dataeval.shift.DriftUnivariate¶

class dataeval.shift.DriftUnivariate(data, method=None, p_val=None, update_strategy=None, correction=None, alternative=None, n_features=None, feature_extractor=None, config=None)¶

Drift detector using univariate statistical tests.

Detects distributional changes by comparing empirical distributions of reference and test datasets using classical statistical tests. For multivariate data, applies the test independently to each feature and aggregates results using multiple testing correction.

Supports five statistical methods with different strengths:

• Kolmogorov-Smirnov (ks): Measures maximum distance between empirical CDFs. General-purpose test, sensitive to middle portions of distributions. Supports directional alternatives for detecting systematic shifts.

• Cramér-von Mises (cvm): Measures integrated squared distance between CDFs. More sensitive than KS to subtle distributional differences across the entire domain. Higher statistical power for many drift types.

• Mann-Whitney U (mwu): Nonparametric rank-based test for stochastic ordering. Robust to outliers and effective for detecting location (median) shifts. Works well with non-normal distributions. Supports directional alternatives.

• Anderson-Darling (anderson): Tests equality of distributions with emphasis on tail differences. More sensitive than KS to heavy-tailed distributions. Ideal for detecting drift in extreme values. Two-sided only.

• Baumgartner-Weiss-Schindler (bws): Modern test emphasizing tail differences with higher power than KS. Balanced sensitivity to both tails and center. Supports directional alternatives. Requires scipy>=1.12.0.

Choosing a Method:

• Use ks for general-purpose drift detection with directional testing

• Use cvm for higher sensitivity to overall distributional changes

• Use mwu for robust detection of median shifts, especially with outliers

• Use anderson when tail behavior is critical (SLA violations, rare events)

• Use bws for best overall power with tail sensitivity and directional testing

Parameters:¶

data : Any¶

Reference dataset used as baseline distribution for drift detection. Can be Array or any type supported by feature_extractor parameter. Should represent the expected data distribution.

method : "ks", "cvm", "mwu", "anderson", or "bws", default "ks"¶

Statistical test method to use. See method descriptions above. Default “ks” provides a well-established general-purpose test.

p_val : float, default 0.05¶

Significance threshold for drift detection, between 0 and 1. Default 0.05 limits false drift alerts to 5% when no drift exists (Type I error rate).

update_strategy : UpdateStrategy or None, default None¶

Strategy for updating reference data when new data arrives. When None, reference data remains fixed throughout detection.

correction : "bonferroni" or "fdr", default "bonferroni"¶

Multiple testing correction method for multivariate drift detection. “bonferroni” provides conservative family-wise error control by dividing significance threshold by number of features. “fdr” uses Benjamini-Hochberg procedure for less conservative control. Default “bonferroni” minimizes false positive drift detections.

alternative : "two-sided", "less" or "greater", default "two-sided"¶

Alternative hypothesis for the statistical test. Applies to: ks, mwu, bws methods only.

• ”two-sided”: detects any distributional difference

• ”less”: tests if test distribution is stochastically smaller

• ”greater”: tests if test distribution is stochastically larger

Default “two-sided” provides most general drift detection. Ignored for cvm and anderson (only support two-sided).

n_features : int | None, default None¶

Number of features to analyze in univariate tests. When None, automatically inferred from the flattened shape of first data sample.

feature_extractor : FeatureExtractor or None, default None¶

Optional feature extraction function to convert input data to arrays. When provided, enables drift detection on non-array inputs such as datasets, metadata, or raw model outputs. The extractor is applied to both reference and test data before drift detection. When None, data must already be Array-like.

config : DriftUnivariate.Config or None, default None¶

Optional configuration object with default parameters. Parameters specified directly in __init__ will override config defaults.

Example

Basic drift detection with Kolmogorov-Smirnov test

>>> train_emb = np.ones((100, 128), dtype=np.float32)
>>> drift_detector = DriftUnivariate(train_emb, method="ks")
>>> test_emb = np.zeros((20, 128), dtype=np.float32)
>>> result = drift_detector.predict(test_emb)
>>> print(f"Drift detected: {result.drifted}")
Drift detected: True

Using Mann-Whitney U for robust median shift detection

>>> drift_mwu = DriftUnivariate(train_emb, method="mwu")
>>> result = drift_mwu.predict(test_emb)

Using Anderson-Darling for tail-sensitive detection

>>> drift_anderson = DriftUnivariate(train_emb, method="anderson")
>>> result = drift_anderson.predict(test_emb)

Detect if test data has systematically higher values

>>> drift_greater = DriftUnivariate(train_emb, method="ks", alternative="greater")
>>> result = drift_greater.predict(test_emb)

Using Baumgartner-Weiss-Schindler with high power

>>> drift_bws = DriftUnivariate(train_emb, method="bws")
>>> result = drift_bws.predict(test_emb)

Access feature-level results

>>> n_features = result.feature_drift
>>> print(f"Features showing drift: {n_features.sum()} / {len(n_features)}")
Features showing drift: 128 / 128

Using configuration:

>>> config = DriftUnivariate.Config(method="cvm", p_val=0.01, correction="fdr")
>>> drift = DriftUnivariate(train_emb, config=config)

predict(data)¶

Predict drift and update reference data using specified strategy.

Performs feature-wise drift detection, applies multiple testing correction, and optionally updates the reference dataset based on the configured update strategy.

Parameters:¶

data : Array¶

Test dataset to analyze for drift against reference data.

Returns:¶

Complete drift detection results including overall drift prediction, corrected thresholds, feature-level analysis, and summary statistics.

Return type:¶

DriftOutput

score(data)¶

Calculate feature-wise p-values and test statistics.

Applies the detector’s statistical test independently to each feature, comparing the distribution of each feature between reference and test data.

Parameters:¶

data : Array¶

Test dataset to compare against reference data.

Returns:¶

First array contains p-values for each feature (all between 0 and 1). Second array contains test statistics for each feature (all >= 0). Both arrays have shape (n_features,).

Return type:¶

tuple[NDArray[np.float32], NDArray[np.float32]]

Notes

Lower p-values indicate stronger evidence of drift for that feature. Higher test statistics indicate greater distributional differences.

property n_features : int¶

Number of features in the reference data.

Lazily computes the number of features from the first data sample if not provided during initialization. Features correspond to the flattened dimensionality of the input data (e.g., pixels for images).

Returns:¶

Number of features (flattened dimensions) in the reference data. Always > 0 for valid datasets.

Return type:¶

int

Notes

For image data, this equals C x H x W. Computed once and cached for efficiency.

property x_ref : numpy.typing.NDArray[numpy.float32]¶

Reference data for drift detection.

Lazily encodes the reference dataset on first access. Data is flattened and converted to 32-bit floating point for consistent numerical processing across different input types.

Returns:¶

Reference data as flattened 32-bit floating point array. Shape is (n_samples, n_features_flattened).

Return type:¶

NDArray[np.float32]

Notes

Data is cached after first access to avoid repeated encoding overhead.

Classes¶