energy_fault_detector.utils.data_downloads

download_backblaze_data(names, dest='./data', overwrite=False)[source]

Download Backblaze drive stats archives and collect CSV files.

Downloads specified ‘data_*.zip’ archives from the Backblaze drive stats page, extracts them to a temporary location, removes macOS metadata directories, and moves all CSV files into a single destination directory.

The final destination directory contains only the CSV files (no ZIP files or _MACOSX directories generated by this function).

Parameters:

  • Union[List[str] (names) – One or more archive identifiers corresponding to ‘data_<name>.zip’ on the Backblaze page (e.g., “Q1_2020”, “Q2_2020”, or “data_Q1_2020.zip”).

  • str] – One or more archive identifiers corresponding to ‘data_<name>.zip’ on the Backblaze page (e.g., “Q1_2020”, “Q2_2020”, or “data_Q1_2020.zip”).

  • dest (Union[str, Path]) – Destination directory where CSV files will be stored.

  • overwrite (bool) – If True, existing CSV files in dest with the same name will be overwritten. If False, existing files are left unchanged.

Returns:

The absolute path to the destination directory containing the CSV files.

Return type:

Path

Raises:

  • ValueError – If no valid archive identifiers are provided.

  • RuntimeError – If no CSV files could be downloaded from the requested archives.

  • requests.HTTPError – If fetching the Backblaze index page or a ZIP file fails.

  • zipfile.BadZipFile – If an archive is invalid.

  • OSError – If filesystem operations fail.

download_zenodo_data(identifier='10.5281/zenodo.15846963', dest='./downloads', remove_zip=True, overwrite=False, flatten_file_structure=True, expected_file_types='*.csv')[source]

Download a Zenodo record via API and unzip any .zip files.

Downloads all files associated with a given Zenodo record (by ID, DOI, or URL), saves them to a local directory, and optionally flattens nested directories that result from extracting ZIP archives.

Parameters:

  • identifier (str) – Zenodo record ID, DOI (e.g., 10.5281/zenodo.15846963), or record URL. Defaults to the CARE2Compare dataset.

  • dest (Path) – Local output directory to save downloaded files. (default: downloads)

  • remove_zip (bool) – If True, ZIP archives will be removed after extraction.

  • overwrite (bool) – If True and dest already exists, contents of dest will be overwritten. Default is False.

  • flatten_file_structure (bool) – If True and unzipping results in a single top-level folder with no conflicting root-level files matching expected_file_types, moves its contents up one level. Default is True.

  • expected_file_types (Union[List[str], str]) – Glob pattern(s) used to detect existing relevant files at the root. If any match, flattening is skipped. Can be a string like ‘.csv’ or list like [’.csv’, ‘.json’]. Default is ‘.csv’.

Returns:

The absolute path to the directory containing the downloaded and unzipped data.

Return type:

Path