Out-of-Distribution Detection

Tutorials

Check out this tutorial to begin using the Out-of-Distribution Detection class

Out-of-Distribution Detection Tutorial

How To Guides

  1. Out-of-Distribution Detection How To

DAML API

class daml.detectors.OOD_AE(model: AE)
fit(x_ref: ~numpy.ndarray, threshold_perc: float = 100.0, loss_fn: ~typing.Callable = <keras.src.losses.MeanSquaredError object>, optimizer: ~keras.src.optimizers.optimizer.Optimizer = <class 'keras.src.optimizers.adam.Adam'>, epochs: int = 20, batch_size: int = 64, verbose: bool = True) → None

Train the AE model with recommended loss function and optimizer.

Parameters:

  • x_ref (np.ndarray) – Training batch.

  • threshold_perc (float, default 100.0) – Percentage of reference data that is normal.

  • loss_fn (Callable, default keras.losses.MeanSquaredError()) – Loss function used for training.

  • optimizer (keras.optimizers.Optimizer, default keras.optimizers.Adam) – Optimizer used for training.

  • epochs (int, default 20) – Number of training epochs.

  • batch_size (int, default 64) – Batch size used for training.

  • verbose (bool, default True) – Whether to print training progress.

predict(X: ndarray, batch_size: int = 10000000000, ood_type: Literal['feature', 'instance'] = 'instance') → Dict[str, ndarray]

Predict whether instances are out-of-distribution or not.

Parameters:

  • X – Batch of instances.

  • ood_type – Predict out-of-distribution at the ‘feature’ or ‘instance’ level.

  • batch_size – Batch size used when making predictions with the autoencoder.

Return type:

Dictionary containing the outlier predictions and both feature and instance level outlier scores.

score(X: ndarray, batch_size: int = 10000000000) → OODScore

Compute instance and (optionally) feature level outlier scores.

Parameters:

  • X (np.ndarray) – Batch of instances.

  • batch_size (int, default int(1e10)) – Batch size used when making predictions with the autoencoder.

Return type:

Instance and feature level outlier scores.

class daml.detectors.OOD_AEGMM(model: AEGMM)
fit(x_ref: ~numpy.ndarray, threshold_perc: float = 100.0, loss_fn: ~typing.Callable = <daml.models.tensorflow.LossGMM object>, optimizer: ~keras.src.optimizers.optimizer.Optimizer = <class 'keras.src.optimizers.adam.Adam'>, epochs: int = 20, batch_size: int = 64, verbose: bool = True) → None

Train the AEGMM model with recommended loss function and optimizer.

Parameters:

  • x_ref (np.ndarray) – Training batch.

  • threshold_perc (float, default 100.0) – Percentage of reference data that is normal.

  • loss_fn (Callable, default LossGMM()) – Loss function used for training.

  • optimizer (keras.optimizers.Optimizer, default keras.optimizers.Adam) – Optimizer used for training.

  • epochs (int, default 20) – Number of training epochs.

  • batch_size (int, default 64) – Batch size used for training.

  • verbose (bool, default True) – Whether to print training progress.

predict(X: ndarray, batch_size: int = 10000000000, ood_type: Literal['feature', 'instance'] = 'instance') → Dict[str, ndarray]

Predict whether instances are out-of-distribution or not.

Parameters:

  • X – Batch of instances.

  • ood_type – Predict out-of-distribution at the ‘feature’ or ‘instance’ level.

  • batch_size – Batch size used when making predictions with the autoencoder.

Return type:

Dictionary containing the outlier predictions and both feature and instance level outlier scores.

score(X: ndarray, batch_size: int = 10000000000) → OODScore

Compute instance and (optionally) feature level outlier scores.

Parameters:

  • X (np.ndarray) – Batch of instances.

  • batch_size (int, default int(1e10)) – Batch size used when making predictions with the autoencoder.

Return type:

Instance and feature level outlier scores.

class daml.detectors.OOD_LLR(model: PixelCNN, model_background: PixelCNN | None = None, log_prob: Callable | None = None, sequential: bool = False)
fit(x_ref: ~numpy.ndarray, threshold_perc: float = 100.0, loss_fn: ~typing.Callable | None = None, optimizer: ~keras.src.optimizers.optimizer.Optimizer = <class 'keras.src.optimizers.adam.Adam'>, epochs: int = 20, batch_size: int = 64, verbose: bool = True, mutate_fn: ~typing.Callable = <function mutate_categorical>, mutate_fn_kwargs: dict = {'feature_range': (0, 255), 'rate': 0.2, 'seed': 0}, mutate_batch_size: int = 10000000000) → None

Train semantic and background generative models.

Parameters:

  • x_ref (np.ndarray) – Training batch.

  • threshold_perc (float, default 100.0) – Percentage of reference data that is normal.

  • loss_fn (Optional[Callable], default None) – Loss function used for training.

  • optimizer (keras.optimizers.Optimizer, default keras.optimizers.Adam) – Optimizer used for training.

  • epochs (int, default 20) – Number of training epochs.

  • batch_size (int, default 64) – Batch size used for training.

  • verbose (bool, default True) – Whether to print training progress.

  • mutate_fn (Callable, default mutate_categorical) – Mutation function used to generate the background dataset.

  • mutate_fn_kwargs (dict, default {"rate": 0.2, "seed": 0, "feature_range": (0, 255)}) – Kwargs for the mutation function used to generate the background dataset. Default values set for an image dataset.

  • mutate_batch_size (int, default int(1e10)) – Batch size used to generate the mutations for the background dataset.

predict(X: ndarray, batch_size: int = 10000000000, ood_type: Literal['feature', 'instance'] = 'instance') → Dict[str, ndarray]

Predict whether instances are out-of-distribution or not.

Parameters:

  • X – Batch of instances.

  • ood_type – Predict out-of-distribution at the ‘feature’ or ‘instance’ level.

  • batch_size – Batch size used when making predictions with the autoencoder.

Return type:

Dictionary containing the outlier predictions and both feature and instance level outlier scores.

score(X: ndarray, batch_size: int = 10000000000) → OODScore

Compute instance and (optionally) feature level outlier scores.

Parameters:

  • X (np.ndarray) – Batch of instances.

  • batch_size (int, default int(1e10)) – Batch size used when making predictions with the autoencoder.

Return type:

Instance and feature level outlier scores.

class daml.detectors.OOD_VAE(model: VAE, samples: int = 10)
fit(x_ref: ~numpy.ndarray, threshold_perc: float = 100.0, loss_fn: ~typing.Callable = <daml.models.tensorflow.Elbo object>, optimizer: ~keras.src.optimizers.optimizer.Optimizer = <class 'keras.src.optimizers.adam.Adam'>, epochs: int = 20, batch_size: int = 64, verbose: bool = True) → None

Train the VAE model.

Parameters:

  • x_ref (np.ndarray) – Training batch.

  • threshold_perc (float, default 100.0) – Percentage of reference data that is normal.

  • loss_fn (Callable, default Elbo(0.05)) – Loss function used for training.

  • optimizer (keras.optimizers.Optimizer, default keras.optimizers.Adam) – Optimizer used for training.

  • epochs (int, default 20) – Number of training epochs.

  • batch_size (int, default 64) – Batch size used for training.

  • verbose (bool, default True) – Whether to print training progress.

predict(X: ndarray, batch_size: int = 10000000000, ood_type: Literal['feature', 'instance'] = 'instance') → Dict[str, ndarray]

Predict whether instances are out-of-distribution or not.

Parameters:

  • X – Batch of instances.

  • ood_type – Predict out-of-distribution at the ‘feature’ or ‘instance’ level.

  • batch_size – Batch size used when making predictions with the autoencoder.

Return type:

Dictionary containing the outlier predictions and both feature and instance level outlier scores.

score(X: ndarray, batch_size: int = 10000000000) → OODScore

Compute instance and (optionally) feature level outlier scores.

Parameters:

  • X (np.ndarray) – Batch of instances.

  • batch_size (int, default int(1e10)) – Batch size used when making predictions with the autoencoder.

Return type:

Instance and feature level outlier scores.

class daml.detectors.OOD_VAEGMM(model: VAEGMM, samples: int = 10)
fit(x_ref: ~numpy.ndarray, threshold_perc: float = 100.0, loss_fn: ~typing.Callable = <daml.models.tensorflow.LossGMM object>, optimizer: ~keras.src.optimizers.optimizer.Optimizer = <class 'keras.src.optimizers.adam.Adam'>, epochs: int = 20, batch_size: int = 64, verbose: bool = True) → None

Train the AE model with recommended loss function and optimizer.

Parameters:

  • X (np.ndarray) – Training batch.

  • threshold_perc (float, default 100.0) – Percentage of reference data that is normal.

  • loss_fn (Callable, default LossGMM(elbo=Elbo(0.05))) – Loss function used for training.

  • optimizer (keras.optimizers.Optimizer, default keras.optimizers.Adam) – Optimizer used for training.

  • epochs (int, default 20) – Number of training epochs.

  • batch_size (int, default 64) – Batch size used for training.

  • verbose (bool, default True) – Whether to print training progress.

predict(X: ndarray, batch_size: int = 10000000000, ood_type: Literal['feature', 'instance'] = 'instance') → Dict[str, ndarray]

Predict whether instances are out-of-distribution or not.

Parameters:

  • X – Batch of instances.

  • ood_type – Predict out-of-distribution at the ‘feature’ or ‘instance’ level.

  • batch_size – Batch size used when making predictions with the autoencoder.

Return type:

Dictionary containing the outlier predictions and both feature and instance level outlier scores.

score(X: ndarray, batch_size: int = 10000000000) → OODScore

Compute instance and (optionally) feature level outlier scores.

Parameters:

  • X (np.ndarray) – Batch of instances.

  • batch_size (int, default int(1e10)) – Batch size used when making predictions with the autoencoder.

Return type:

Instance and feature level outlier scores.

class daml.detectors.OODScore(instance_score: ndarray, feature_score: ndarray | None = None)

NamedTuple containing the instance and (optionally) feature score.

Parameters:

  • instance_score (np.ndarray) – Instance score of the evaluated dataset.

  • feature_score (Optional[np.ndarray], default None) – Feature score, if available, of the evaluated dataset.