CARE2Compare guide

Note

This page documents how CARE2Compare support is exposed in EnergyFaultDetector. For interpretation and recurring dataset questions, see CARE2Compare FAQ.

EnergyFaultDetector provides support for:

  • loading CARE2Compare event datasets,

  • accessing event metadata,

  • formatting normal-operation masks based on status_type_id,

  • evaluating predictions with the CARE score.

Relevant classes include:

Background

The CARE2Compare dataset and CARE score are introduced in:

CARE to Compare: A Real-World Benchmark Dataset for Early Fault Detection in Wind Turbine Data

In the package, CARE2Compare support is intended to help with two tasks:

  1. loading event-based benchmark datasets,

  2. evaluating anomaly predictions in a way that is aligned with the CARE benchmark.

Conceptual overview

The dataset contains two related but distinct kinds of labels:

  • status_type_id: timestamp-level operating-status information,

  • event_label: event-level label indicating whether the prediction section contains an anomalous event.

These labels should not be treated as interchangeable.

In particular, anomalous events may still contain timestamps with status_type_id = 0. This is expected for CARE2Compare and reflects the difference between:

  • the operator’s recorded turbine status at a timestamp, and

  • the retrospectively defined anomaly window used for early fault detection.

For details, see CARE2Compare FAQ.

Dataset loading

The energy_fault_detector.evaluation.care2compare.Care2CompareDataset helper can be used to load local CARE2Compare data or download the dataset automatically, depending on package configuration.

Typical usage:

from energy_fault_detector.evaluation.care2compare import Care2CompareDataset

dataset = Care2CompareDataset(
    path="./CARE_To_Compare",
    download_dataset=False,
)

x_train, x_test = dataset.load_event_dataset(event_id=53, index_column="time_stamp")
info = dataset.get_event_info(53)

The returned event metadata can then be used for evaluation, filtering, or reporting.

Formatted loading

If you want a convenience split into values and normal-operation masks, use the formatted loader.

x_train, train_normal, x_test, test_normal = dataset.load_and_format_event_dataset(event_id=53)

In the package, the normal masks are derived from:

status_type_id == 0

This is mainly useful when training normal-behavior models or when applying pointwise evaluation logic.

Training-data filtering

For normal-behavior modeling, it is generally recommended to filter out timestamps that are clearly not representative of normal operation.

In practice, this often means using status_type_id to exclude abnormal operating modes from the training data.

However, CARE2Compare should not be understood as fully pre-cleaned data. Depending on the model and wind farm, additional preprocessing may still be needed, such as:

  • missing-value handling,

  • invalid-measurement filtering,

  • feature selection,

  • angle transformation,

  • scaling.

Example workflow

A typical workflow looks like this:

  1. load an event dataset,

  2. separate training and prediction sections,

  3. derive or load model predictions for prediction timestamps,

  4. evaluate those predictions with energy_fault_detector.evaluation.care_score.CAREScore.

Example:

import pandas as pd
from energy_fault_detector.evaluation.care2compare import Care2CompareDataset
from energy_fault_detector.evaluation.care_score import CAREScore

dataset = Care2CompareDataset(path="./CARE_To_Compare", download_dataset=False)
scorer = CAREScore()

# Iterate over the events of a chosen wind farm. The data is loaded with
# timestamps as the index so they match the event_start / event_end metadata.
for x_train, train_normal, x_test, test_normal, event_id in dataset.iter_formatted_datasets(
    wind_farm="B", index_column="time_stamp",
):
    info = dataset.get_event_info(event_id)

    # Example placeholder prediction: one boolean anomaly prediction per
    # timestamp in the prediction section, indexed like x_test.
    predicted_anomalies = pd.Series(False, index=x_test.index)

    scorer.evaluate_event(
        event_start=info["event_start"],
        event_end=info["event_end"],
        event_label=info["event_label"],
        predicted_anomalies=predicted_anomalies,
        normal_index=test_normal,
        event_id=event_id,
    )

final_score = scorer.get_final_score()

Note

get_final_score requires at least one evaluated anomaly event and one evaluated normal event. A single evaluate_event call is fine for a quick smoke test, but the final CARE score can only be computed once events of both labels have been added.

CARE score overview

The CARE score combines four aspects of fault detection quality:

  • Coverage

  • Accuracy

  • Reliability

  • Earliness

At a high level:

  • Coverage and Earliness are computed on anomalous events,

  • Accuracy is computed on normal events,

  • Reliability is an event-wise score based on event decisions.

The package implementation follows the CARE logic provided in the publication and subsequent package support. For detailed interpretation notes, see CARE2Compare FAQ.

Wind-farm-specific interpretation

The meaning and usefulness of status_type_id differs slightly between wind farms.

For Wind Farms B and C, status labels reflect anonymized operator-provided status information and are useful both for training-data filtering and for parts of CARE-style evaluation.

For Wind Farm A, the status labels should mainly be used for training-data filtering. For prediction-time CARE evaluation, they should largely be ignored; pass ignore_normal_index=True to evaluate_event() so that every timestamp is evaluated regardless of status_type_id.

This distinction is important when interpreting evaluation results.

Known practical limitations

Users should be aware of several dataset and benchmark limitations:

  • the dataset is anonymized,

  • the data is not fully preprocessed for modeling,

  • exact benchmark code from the original paper is not guaranteed to match the current package state,

  • overlapping timestamps across different event files and and no temporal order of events from the same asset may occur due to anonymization,

  • some version-specific data-quality notes apply.

These are discussed in more detail in CARE2Compare FAQ.

See also