dataeval.performance.SufficiencyOutput¶

class dataeval.performance.SufficiencyOutput¶

Output class for Sufficiency workflow.

steps¶

Array of sample sizes

Type:¶: NDArray

measures¶

3D array [runs, substep, classes] of values for all runs observed for each sample size step for each measure

Type:¶: dict[str, NDArray]

averaged_measures¶

Average of values for all runs observed for each sample size step for each measure

Type:¶: dict[str, NDArray]

unit_interval¶

Constrains the power law to the interval [0, 1]. Set True (default) for metrics such as accuracy, precision, and recall which are defined to take values on [0,1]. Set False for metrics not on the unit interval.

Type:¶: bool, default True

data()¶

Return the output data as a dictionary.

Return type:¶: dict[str, Any]

get_params(n_iter=1000)¶

Get the curve fit parameters for the power law model.

Parameters:¶

n_iter : int, default 1000¶: Number of iterations to perform in the basin-hopping curve-fit process

Returns:¶

Mapping of metric names to their fitted power law parameters

Return type:¶

Mapping[str, NDArray]

inv_project(targets, n_iter=1000)¶

Calculate training samples needed to achieve target metric values.

Parameters:¶

targets : Mapping[str, ArrayLike] | ArrayLike¶: Mapping of target metric scores (from 0.0 to 1.0) that we want to achieve, where the key is the name of the metric. If an ArrayLike is provided, the same targets are applied to all metrics.
n_iter : int, default 1000¶: Number of iterations to perform in the basin-hopping curve-fit process

Returns:¶

DataFrame with columns:: target: float - Target metric value <metric>: int - One column per metric (multi-class as metric_0, metric_1, …) Values of -1 indicate the target is unachievable.

Return type:¶

pl.DataFrame

meta()¶

Metadata about the execution of the function or method for the Output class.

Return type:¶: ExecutionMetadata

project(projection, n_iter=1000)¶

Project metric values to new sample sizes using the fitted power law.

Parameters:¶

projection : int | Iterable[int]¶: Sample size(s) to project metric values to
n_iter : int, default 1000¶: Number of iterations to perform in the basin-hopping curve-fit process

Returns:¶

DataFrame with columns:: step: int - The projected sample size <metric>: float - One column per metric (multi-class as metric_0, metric_1, …)

Return type:¶

pl.DataFrame

Raises:¶

ValueError – If projection is not numerical

to_dataframe()¶

Convert averaged measures to a DataFrame.

Returns:¶: DataFrame with step column and one column per metric.
Return type:¶: pl.DataFrame

property params : collections.abc.Mapping[str, numpy.typing.NDArray[numpy.float64]]¶

Get the curve fit parameters for the power law model.

Returns:¶: Mapping of metric names to their fitted power law parameters
Return type:¶: Mapping[str, NDArray]