energy_fault_detector

The energy-fault-detector package

class Config(config_filename=None, config_dict=None)[source]

Bases: BaseConfig

Configuration class. Either config_filename or config_dict must be provided. Reads a yaml file with the anomaly detection configuration and sets corresponding settings.

property arcana_params: Dict[str, Any]

Get the ARCANA parameters.

property data_clipping: bool

Whether to clip training data.

property data_clipping_params: Dict[str, Any]

Data clipping parameters.

property data_preprocessor_steps: List[Dict[str, Any]]

Get the data preprocessor steps.

property data_split_params: Dict[str, Any]

DataSplitter or train_test_split parameters.

property dtype

Data type, float32 by default.

property fit_threshold_on_val: bool

Whether to fit threshold on validation data only.

property max_criticality: int | None

Max criticality value.

property protect_conditional_features: bool

Whether to protect conditional features from being dropped by preprocessing.

property root_cause_analysis: bool

Whether to run ARCANA.

class FaultDetector(config=None, model_directory='fault_detector_model')[source]

Bases: FaultDetectionModel

Main class for fault detection in renewable energy assets and power grids.

Parameters:

  • config (Optional[Config]) – Config object with fault detection configuration. Defaults to None. If None, the models need to be loaded from a path using the load_models method.

  • model_directory (str | Path) – Directory to save models to. Defaults to ‘fault_detector_model’.

anomaly_score

AnomalyScore object.

autoencoder

Autoencoder object.

threshold_selector

ThresholdSelector object.

data_preprocessor

DataPreprocessorPipeline object.

save_timestamps

a list of string timestamps indicating when the model was saved.

fit(sensor_data, normal_index=None, save_models=True, overwrite_models=False, fit_autoencoder_only=False, fit_preprocessor=True, **kwargs)[source]

Fit models on the given sensor_data and save them locally and return the metadata.

Return type:

ModelMetadata

predict(sensor_data, model_path=None, root_cause_analysis=False, track_losses=False, track_bias=False)[source]

Predict with given models for a specific asset

Parameters:

  • sensor_data (DataFrame) – DataFrame with the sensor data of one asset for a specific time window. The timestamp should be the index and the sensor values as columns.

  • model_path (Optional[str]) – Path to the models to be applied. If None, assumes the attributes data_preprocessor, autoencoder, anomaly_score, and threshold_selector contain fitted instances.

  • root_cause_analysis (bool) – Whether to run ARCANA. Defaults to False.

  • track_losses (bool) – Optional; if True, ARCANA losses will be tracked over the iterations. Defaults to False.

  • track_bias (bool) – Optional; if True, ARCANA bias will be tracked over the iterations. Defaults to False.

Returns:

with the following DataFrames:

  • predicted_anomalies: Series with detected anomalies (bool).

  • reconstruction: DataFrame with reconstruction of the sensor data with timestamp as index.

  • recon_error: DataFrame with reconstruction errors.

  • anomaly_score: Series with anomaly scores for each timestamp.

  • bias_data: DataFrame with ARCANA results with timestamp as index. None if ARCANA was not run.

  • arcana_losses: DataFrame containing recorded values for all losses in ARCANA. None if ARCANA was not run.

  • tracked_bias: List of DataFrames. None if ARCANA was not run.

Return type:

FaultDetectionResult

predict_anomalies(scores, x_prepped=None)[source]

Predict anomalies based on anomaly scores.

Return type:

Series

predict_anomaly_score(sensor_data)[source]

Predict the anomaly score.

Return type:

Series

preprocess_train_data(sensor_data, normal_index, fit_preprocessor=True)[source]

Preprocesses the training data using the configured data_preprocessor.

Return type:

Tuple[DataFrame, DataFrame, Series]

run_root_cause_analysis(sensor_data, track_losses=False, track_bias=False)[source]

Run ARCANA

Parameters:

  • sensor_data (DataFrame) – pandas DataFrame containing the sensor data which should be analyzed.

  • track_losses (bool) – optional bool. If True the arcana losses will be tracked over the iterations

  • track_bias (bool) – optional bool. If True the arcana bias will be tracked over the iterations

Return type:

Tuple[DataFrame, DataFrame, List[DataFrame]]

Returns: Tuple of (pd.DataFrame, pd.DataFrame, List[pd.DataFrame])

df_arcana_bias: pandas dataframe containing the arcana bias. arcana_losses: dictionary containing loss names as keys and lists of loss values as values. tracked_bias: list of pandas dataframe containing the arcana bias recorded over the iterations.

tune(sensor_data, normal_index=None, pretrained_model_path=None, new_learning_rate=0.0001, tune_epochs=10, tune_method='full', save_models=True, overwrite_models=False, data_preprocessor=None)[source]
FaultDetector finetuning via the following methods:

‘full’ (all autoencoder weights + threshold and anomaly-score scaling will be adapted), ‘decoder’ (only decoder weights + threshold will be adapted), ‘threshold’ (only the threshold and anomaly-score scaling is adapted)

Notes: Parameters tune_epochs and new_learning_rate should be chosen carefully while considering

potential overfitting issues depending on the similarity of the tuning data and the training data.

Parameters:

  • sensor_data (DataFrame) – DataFrame with the sensor data of one asset for a specific time window. The timestamp should be the index and the sensor values as columns.

  • normal_index (Optional[Series]) – Series indicating normal behavior as boolean with the timestamp as index. If not provided, it is assumed all data in sensor_data represents normal behaviour. Defaults to None.

  • pretrained_model_path (Optional[str]) – Path to pretrained model. If None, assumes attributes data_preprocessor, autoencoder, anomaly_score, and threshold_selector contain fitted instances.

  • tune_epochs (int) – Number of epochs to fine-tune. Defaults to 10.

  • new_learning_rate (float) – Learning rate to tune the autoencoder with. Defaults to 0.0001.

  • tune_method (str) – Possible options: ‘full’ (all autoencoder weights + threshold and anomaly-score scaling will be adapted), ‘decoder’ (only decoder weights + threshold will be adapted), ‘threshold’ (only the threshold and anomaly-score scaling is adapted) Defaults to ‘full’.

  • save_models (bool) – Whether to save models. Defaults to True.

  • overwrite_models (bool) – If True, existing model directories can be overwritten. Defaults to False.

  • data_preprocessor (Optional[DataPreprocessor]) – Optional prefitted data preprocessor. Useful when using a generic preprocessor for all models.

Returns:

metadata of the trained model with model_date, model_path, model reconstruction errors of the training and validation data.

Return type:

ModelMetadata