dataeval.metrics.estimators

Estimators calculate performance bounds and the statistical distance between datasets.

Output Classes

BEROutput

Output class for ber() estimator metric.

ClustererOutput

Output class for clusterer().

DivergenceOutput

Output class for divergence() estimator metric.

NullModelMetricsOutput

Output class for null-model metrics

UAPOutput

Output class for uap() estimator metric.

Functions

ber(embeddings, labels[, k, method])

An estimator for Multi-class Bayes error rate using FR or KNN test statistic basis.

clusterer(data)

Uses hierarchical clustering on the flattened data and returns clustering

divergence(emb_a, emb_b[, method])

Calculates the divergence by counting the number of "between dataset" edges in the

null_model_metrics(test_labels[, train_labels])

Calculate null model metrics (dummy classifiers metrics) for given class distributions.

uap(labels, scores)

FR Test Statistic based estimate of the empirical mean precision for the upperbound average precision.