energy_fault_detector

Autoencoder-based anomaly and fault detection for renewable energy assets and power grids.

The package learns a normal-behaviour model from operational data and flags deviations as anomalies, and includes the ARCANA method for root-cause analysis of detected anomalies.

class Config(config_filename=None, config_dict=None)[source]

Bases: BaseConfig

Configuration class. Either config_filename or config_dict must be provided. Reads a yaml file with the anomaly detection configuration and sets corresponding settings.

property arcana_params: Dict[str, Any]

Get the ARCANA parameters.

property data_clipping: bool

Whether to clip training data.

property data_clipping_params: Dict[str, Any]

Data clipping parameters.

property data_preprocessor_steps: List[Dict[str, Any]]

Get the data preprocessor steps.

property data_split_params: Dict[str, Any]

DataSplitter or train_test_split parameters.

property dtype

Data type, float32 by default.

property fit_threshold_on_val: bool

Whether to fit threshold on validation data only.

property max_criticality: int | None

Max criticality value.

property protect_conditional_features: bool

Whether to protect conditional features from being dropped by preprocessing.

property root_cause_analysis: bool

Whether to run ARCANA.

class FaultDetector(config=None, model_directory='fault_detector_model')[source]

Bases: FaultDetectionModel

Main class for fault detection in renewable energy assets and power grids.

Parameters:

  • config (Optional[Config]) – Config object with fault detection configuration. Defaults to None. If None, the models need to be loaded from a path using the load_models method.

  • model_directory (str | Path) – Directory to save models to. Defaults to ‘fault_detector_model’.

anomaly_score

AnomalyScore object.

autoencoder

Autoencoder object.

threshold_selector

ThresholdSelector object.

data_preprocessor

DataPreprocessorPipeline object.

save_timestamps

a list of string timestamps indicating when the model was saved.

fit(sensor_data, normal_index=None, save_models=True, overwrite_models=False, fit_autoencoder_only=False, fit_preprocessor=True, **kwargs)[source]

Fit models on the given sensor_data and save them locally and return the metadata.

Return type:

ModelMetadata

predict(sensor_data, model_path=None, root_cause_analysis=False, track_losses=False, track_bias=False)[source]

Predict with given models for a specific asset

Parameters:

  • sensor_data (DataFrame) – DataFrame with the sensor data of one asset for a specific time window. The timestamp should be the index and the sensor values as columns.

  • model_path (Optional[str]) – Path to the models to be applied. If None, assumes the attributes data_preprocessor, autoencoder, anomaly_score, and threshold_selector contain fitted instances.

  • root_cause_analysis (bool) – Whether to run ARCANA. Defaults to False.

  • track_losses (bool) – Optional; if True, ARCANA losses will be tracked over the iterations. Defaults to False.

  • track_bias (bool) – Optional; if True, ARCANA bias will be tracked over the iterations. Defaults to False.

Returns:

with the following DataFrames:

  • predicted_anomalies: Series with detected anomalies (bool).

  • reconstruction: DataFrame with reconstruction of the sensor data with timestamp as index.

  • recon_error: DataFrame with reconstruction errors.

  • anomaly_score: Series with anomaly scores for each timestamp.

  • bias_data: DataFrame with ARCANA results with timestamp as index. None if ARCANA was not run.

  • arcana_losses: DataFrame containing recorded values for all losses in ARCANA. None if ARCANA was not run.

  • tracked_bias: List of DataFrames. None if ARCANA was not run.

Return type:

FaultDetectionResult

predict_anomalies(scores, x_prepped=None)[source]

Predict anomalies based on anomaly scores.

Return type:

Series

predict_anomaly_score(sensor_data)[source]

Predict the anomaly score.

Return type:

Series

preprocess_train_data(sensor_data, normal_index, fit_preprocessor=True)[source]

Preprocesses the training data using the configured data_preprocessor. If categorical features are declared in config file, data is split into categorical and numerical features. Boolean features are treated the same as numerical features. Encoded categorical features are scaled or not depending on the config flag.

Returns:

Preprocessed training data after filtering only normal index, original sensor data (clipped if data clipping is enabled), and normal index.

Return type:

Tuple[DataFrame, DataFrame, Series]

run_root_cause_analysis(sensor_data, track_losses=False, track_bias=False)[source]

Run ARCANA

Parameters:

  • sensor_data (DataFrame) – pandas DataFrame containing the sensor data which should be analyzed.

  • track_losses (bool) – optional bool. If True the arcana losses will be tracked over the iterations

  • track_bias (bool) – optional bool. If True the arcana bias will be tracked over the iterations

Return type:

Tuple[DataFrame, DataFrame, List[DataFrame]]

Returns: Tuple of (pd.DataFrame, pd.DataFrame, List[pd.DataFrame])

df_arcana_bias: pandas dataframe containing the arcana bias. arcana_losses: dictionary containing loss names as keys and lists of loss values as values. tracked_bias: list of pandas dataframe containing the arcana bias recorded over the iterations.

tune(sensor_data, normal_index=None, pretrained_model_path=None, new_learning_rate=0.0001, tune_epochs=10, tune_method='full', save_models=True, overwrite_models=False, data_preprocessor=None)[source]
FaultDetector finetuning via the following methods:

‘full’ (all autoencoder weights + threshold and anomaly-score scaling will be adapted), ‘decoder’ (only decoder weights + threshold will be adapted), ‘threshold’ (only the threshold and anomaly-score scaling is adapted)

Notes: Parameters tune_epochs and new_learning_rate should be chosen carefully while considering

potential overfitting issues depending on the similarity of the tuning data and the training data.

Parameters:

  • sensor_data (DataFrame) – DataFrame with the sensor data of one asset for a specific time window. The timestamp should be the index and the sensor values as columns.

  • normal_index (Optional[Series]) – Series indicating normal behavior as boolean with the timestamp as index. If not provided, it is assumed all data in sensor_data represents normal behaviour. Defaults to None.

  • pretrained_model_path (Optional[str]) – Path to pretrained model. If None, assumes attributes data_preprocessor, autoencoder, anomaly_score, and threshold_selector contain fitted instances.

  • tune_epochs (int) – Number of epochs to fine-tune. Defaults to 10.

  • new_learning_rate (float) – Learning rate to tune the autoencoder with. Defaults to 0.0001.

  • tune_method (str) – Possible options: ‘full’ (all autoencoder weights + threshold and anomaly-score scaling will be adapted), ‘decoder’ (only decoder weights + threshold will be adapted), ‘threshold’ (only the threshold and anomaly-score scaling is adapted) Defaults to ‘full’.

  • save_models (bool) – Whether to save models. Defaults to True.

  • overwrite_models (bool) – If True, existing model directories can be overwritten. Defaults to False.

  • data_preprocessor (Optional[DataPreprocessor]) – Optional prefitted data preprocessor. Useful when using a generic preprocessor for all models.

Returns:

metadata of the trained model with model_date, model_path, model reconstruction errors of the training and validation data.

Return type:

ModelMetadata