neptoon.io.save

save_data¶

Classes:

SaveAndArchiveOutputs
ConfigSaver

Functions:

create_save_folder
close_and_save_data_audit_log
append_hash_to_folder_name
mask_bad_data
save_data_frames
save_outputs
save_custom_column_names
save

SaveAndArchiveOutputs ¶

SaveAndArchiveOutputs(folder_name, processed_data_frame, flag_data_frame, sensor_info, save_folder_location=None, use_custom_column_names=False, custom_column_names_dict=None, append_timestamp=True, figure_handler=None, calib_df=None, magazine_active=False)

Handles saving the outputs from neptoons in an organised way.

Future Ideas:

options to compress outputs (zip_output: bool = True)
cloud connection
bespoke output formats

Attributes

Parameters:

Name	Type	Description	Default
`folder_name`	`str`	Desired name for the save folder	required
`processed_data_frame`	`DataFrame`	The processed time series data	required
`flag_data_frame`	`DataFrame`	The flag dataframe	required
`sensor_info`	`SensorInfo`	The SensorInfo object.	required
`save_folder_location`	`Union[str, Path]`	The folder where the data should be saved. If left as None	`None`
`use_custom_column_names`	`bool`	Whether to use custom column names, by default False	`False`
`custom_column_names_dict`	`dict`	A dictionary to convert standard neptoon names into custom a custom naming convention, by default None	`None`
`append_timestamp`	`bool`	Whether to append a timestamp to the folder name when saving.	`True`

create_save_folder ¶

create_save_folder()

Creates the folder location where the data will be saved.

close_and_save_data_audit_log ¶

close_and_save_data_audit_log(append_hash=False)

NOTE: CURRENTLY NOT IMPLEMENTED

Handles closing the data audit log, producing the YAML output, and optionally appending a hash to the save location folder name.

This function performs the following steps: 1. Archives and deletes the data audit log using DataAuditLog.archive_and_delete_log()

2. If append_hash is True:
    a. Locates the hash.txt file in the data_audit_log
       subfolder
    b. Reads the first 6 characters of the hash
    c. Renames the main folder to include this hash

Parameters:

append_hash : bool, optional (default=False) If True, appends the first 6 characters of the hash from hash.txt to the folder name.

append_hash_to_folder_name ¶

append_hash_to_folder_name(folder_path)

Appends the first 6 characters of the hash from hash.txt to the folder name.

Parameters:

folder_path : pathlib.Path The path to the folder to be renamed.

Returns:

pathlib.Path The path to the renamed folder.

Raises:

FileNotFoundError If the data audit log folder or hash.txt file is not found. PermissionError If permissions to access the folder are not available.

mask_bad_data ¶

mask_bad_data()

Masks out flagged data with nan values

save_data_frames ¶

save_data_frames(file_name)

Saves various data frames as .csv files.

save_outputs ¶

save_outputs(use_custom_column_names=False)

The main function which chains the options.

Create folder
Mask time series
Save time series
Save flag df
Optional: Save bespoke time series
Optional: Save DAL
Optional: Save Journalist
Optional: rename folder
Optional: compress data

save_custom_column_names ¶

save_custom_column_names()

WIP - save custom variable names using ColumnInfo.

ConfigSaver ¶

ConfigSaver(save_folder_location, config)

Saves the SensorConfig object as a yaml file.

save ¶

save()

Convert a Pydantic model to YAML file and save it.