anomaly_scores

Anomaly scores that reduce per-feature reconstruction errors to one score per sample.

After an autoencoder reconstructs its input, an AnomalyScore aggregates the per-feature reconstruction errors into a single scalar per sample. Samples with high scores are candidate anomalies.

Each scorer subclasses AnomalyScore and is selected via the train.anomaly_score config key.

class MahalanobisScore(pca=<object object>, pca_min_var=0.9, mcd_support_fraction=0.9, scale=<object object>, covariance_method='auto', min_cov_det_ratio=2.0, shrinkage_method='oas')[source]

Bases: AnomalyScore

Calculate Mahalanobis scores on the reconstruction errors.

This class implements the mahalanobis norm calculation based on multiple variants of covariance matrix estimation. The cases are defined by the user-set parameter pca and the relation of the number of samples n and the number of features p:

1. (pca=False) and n>=min_cov_det_ratio * p: Estimates the covariance of the mean-centred reconstruction errors using the MinCovDet algorithm and returns the Mahalanobis distance of each sample. 2. (pca=False) and n<min_cov_det_ratio * p: Estimates the covariance of the mean-centred reconstruction errors using shrinkage based algorithms like Ledoit-Wolf or OAS. The estimator can be specified via the parameter covariance_method and automatically falls back to a shrinkage estimator (Ledoit-Wolf/OAS) when MinCovDet would become unstable. 3. (pca=True): A PCA is carried out on standardized reconstruction errors to reduce the dimensionality. MinCovDet is then used to estimate the Covariance matrix in order to compute the Mahalanobis distance of each sample. This method is deprecated because it does not consider all correlations in the reconstruction errors which might lead to less useful anomaly scores.

Deprecated since version The: pca=True path is deprecated and will be replaced by full covariance matrix estimators. It is kept for backwards compatibility; use pca=False for the direct full-dimensionality Mahalanobis distance. The scale parameter is deprecated and no longer has any effect.

Parameters:

  • pca (bool) – deprecated. If True, standardize, reduce dimensionality via PCA and estimate the covariance with MinCovDet (legacy behaviour). If False, estimate the covariance on the full-dimensionality mean-centred errors. Default True.

  • scale (bool) – deprecated, no longer has any effect.

  • pca_min_var (float) – parameter for PCA, variance to keep. Default 0.9

  • mcd_support_fraction (float) – parameter for Minimum Covariance Determinant estimation. Default 0.9

  • covariance_method (str) –

    which covariance estimator to use in the (pca=False) path. One of:

    • "auto" (default): use MinCovDet when n / p >= min_cov_det_ratio and the resulting covariance is stable (full rank / well conditioned), otherwise fall back to a shrinkage estimator (shrinkage_method).

    • "min_cov_det": always use MinCovDet. May raise or be unstable when n is not sufficiently larger than p.

    • "shrinkage": always use a shrinkage estimator (shrinkage_method).

  • min_cov_det_ratio (float) – minimum sample-to-feature ratio n / p required (in "auto" mode) for MinCovDet to be used instead of shrinkage. Default 2.0.

  • shrinkage_method (str) – which shrinkage estimator to use. One of "oas" (default) or "ledoit_wolf".

Configuration example:

train:
  anomaly_score:
    name: mahalanobis
    params:
      pca: False
      covariance_method: auto
      min_cov_det_ratio: 2.0
      shrinkage_method: oas
fit(x, y=None)[source]

Fit the covariance estimator to determine Mahalanobis distance.

Parameters:

  • x (Union[DataFrame, ndarray]) – numpy 2d array or pandas DataFrame with differences between prediction and actual sensor values.

  • y (Optional[Series]) – not used, labels indicating whether sample is normal (True) or anomalous (False).

Return type:

MahalanobisScore

standardize(x)[source]

Standardization of the reconstruction error in x (only used in the deprecated PCA path).

transform(x)[source]

Calculate Mahalanobis distance from x.

Parameters:

x (Union[DataFrame, ndarray]) – numpy 2d array or pandas Dataframe with differences between prediction and actual sensor values

Return type:

Series

Returns:

Mahalanobis distance for each sample. Output is a pandas Series if input was a pandas DataFrame

class RMSEScore(scale=<object object>, **kwargs)[source]

Bases: AnomalyScore

Calculate the RMSE of given reconstruction errors.

Deprecated since version The: scale parameter no longer has any effect and is only accepted for backwards compatibility. It will be removed in a future version.

Configuration example:

train:
  anomaly_score:
    name: rmse
fit(x, y=None)[source]

Calculate standard deviation and mean on training data

Parameters:

  • x (Union[DataFrame, ndarray]) – numpy 2d array with differences between prediction and actual sensor values

  • y (Optional[Series]) – not used, labels indicating whether sample is normal (True) or anomalous (False).

Return type:

RMSEScore

standardize(x)[source]

Standardization of the reconstruction error in x

transform(x)[source]

Calculate the RMSE based on the deviation matrix.

Parameters:

x (Union[DataFrame, ndarray]) – numpy 2d array or pandas Dataframe with differences between prediction and actual sensor values

Return type:

Series

Returns:

RMSE for each sample.