dataeval.metrics.estimators

Estimators calculate performance bounds and the statistical distance between datasets.

Output Classes

BEROutput

Output class for ber() estimator metric.

DivergenceOutput

Output class for divergence() estimator metric.

NullModelMetricsOutput

Output class for null-model metrics

UAPOutput

Output class for uap() estimator metric.

Functions

ber(embeddings, labels[, k, method])

An estimator for Multi-class Bayes error rate using FR or KNN test statistic basis.

divergence(emb_a, emb_b[, method])

Calculates the divergence by counting the number of "between dataset" edges in the

null_model_metrics(test_labels[, train_labels])

Calculate null model metrics (dummy classifiers metrics) for given class distributions.

uap(labels, scores)

FR Test Statistic based estimate of the empirical mean precision for the upperbound average precision.